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SUMMARY 


In  this  chapter  we  analyze  the  interaction  between  model  simplification 
and  strategy  design  in  a  multimodel  context  and  for  multiple  agent  stochastic 
decision  problems  with  decentralized  information.  Under  quasi-classical 
information  patterns,  and  using  singular  perturbations  approach,  we  establish 
asymptotic  optimality  of  different  multimodels  which  involve  continuous  and 
two  types  of  sampled  measurements.  Our  general  analysis  and  discussion  serve 
to  enhance  our  understanding  of  the  interrelationships  between  structual 
features  of  stochastic  large  scale  systems,  like  time-scales  and  weak  coupling, 
and  strategy  design. 
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1 .  INTRODUCTION 


The  problem  of  efficient  management  and  control  of  large  scale 
systems  has  been  extremely  challenging  to  control  engineers.  There  are 
essentially  two  main  issues  of  concern:  the  modeling  issue  is  complicated 
due  to  the  large  dimension  of  the  system,  and  the  control  design  issue  is 
complicated  due  to  the  presence  of  multiple  decision  makers  having  possibly 
different  goals  and  possessing  decentralized  information.  Efforts  to  under¬ 
stand  the  inherent  complexities  have  led  to  the  concept  of  nonclassical 
information  patterns  [1].  This  concept  expresses  a  basic  fact  that  a  decision 
maker  has  neither  complete  nor  instantaneous  access  to  other  decision  makers' 
measurements  and  decisions.  A  related  but  perhaps  more  basic  fact  is 
expressed  by  the  multimodeling  concept  [2].  This  concept  accounts  for  the 
many  realistic  situations  when  different  decision  makers  have  different  infor¬ 
mation  about  the  system  structure  and  dynamics  and  therefore  use  different 
simplified  models  of  the  same  large  scale  system.  These  models  may  differ  in 
parameter  values,  signal  uncertainties,  and,  more  critically,  in  their  basic 
structural  properties. 

A  strong  motivation  for  the  multimodeling  approach  is  found  in 
multi-area  power  systems.  The  decision  maker  in  one  area  uses  a  detailed 
model  of  his  area  only  and  some  lower  order  "equivalent"  of  the  rest  of  the 
system.  The  decision  makers  in  other  areas  behave  in  a  similar  way  and  as 
a  result  each  has  his  own  view  of  the  same  large  scale  system.  The  main 
advantage  of  such  an  empirical  decomposition  is  that  it  leads  to  distributed 
computations  and  less  communication  between  the  controllers  because  each 
decision  maker  would  only  require  measurements  of  the  variables  appearing  in 
his  own  reduced  order  model.  Many  crucial  problems  (instability,  suboptimality. 
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etc.)  arise  because  the  strategies  designed  with  such  inconsistent  models  are 
then  applied  to  the  actual  system. 

We  investigate,  in  this  chapter,  the  effect  of  multimodeling  incon¬ 
sistencies  on  the  design  and  implementation  of  multicontroller  strategies 
under  certain  quasi-classical  information  patterns.  The  approach  taken  is 
perturbational.  If  the  model  inconsistencies  are  small,  it  is  natural  to 
expect  that  their  effect  on  the  designed  strategies  and  on  the  actual  system 
performance  would  be  in  some  sense  small.  If  this  were  not  the  case,  the 
designed  strategies  would  not  be  applicable  to  realistic  systems  whose  models 
are  never  exactly  known.  We  consider  this  low  sensitivity  property  a 
sine  qua  non  condition  for  any  control  design  and,  in  particular,  for  the 
design  of  large  scale  systems  controlled  from  multiple  control  stations. 

Another  fundamental  property  of  our  perturbational  approach  is  that 
it  concentrates  on  modeling  errors  caused  by  reducing  the  model  order.  Such 
order  reductions  are  achieved  by  separating  the  time  scales,  that  is,  by 
considering  slow  and  fast  phenomena  separately.  A  typical  situation  is  when 
the  decision  maker  in  one  area  neglects  the  fast  phenomena  in  all  other  areas. 
In  geographically  dispersed  systems  this  practice  is  based  on  the  experimental 
observation  that  faster  phenomena  propagate  to  shorter  distances  than  the 
slower  phenomena.  For  example,  in  a  multimachine  transient  the  slower 
oscillatory  modes  are  observed  throughout  the  system,  while  faster  inter¬ 
machine  oscillations  are  of  a  more  local  character  [3]. 

A  tool  for  analyzing  the  change  in  model  order  is  the  so-called 
singular  perturbation  method  which  converts  the  change  of  model  order  into  a 
small  parameter  perturbation  [4].  This  parameter  multiplies  the  derivatives 
of  the  fast  state  variables  and  when  it  is  set  to  zero  the  fast  phenomena  are 


neglected.  The  fast  phenomena  are  treated  separately  in  the  fast  time  scale 
where  the  slow  variables  are  "frozen"  at  their  quasi-steady  state  values. 

This  two-time-scale  approach  is  asymptotic,  that  is,  exact  in  the  limit  as 
the  ratio  of  speeds  of  the  slow  versus  the  fast  dynamics  tends  to  zero.  When 
this  ratio  is  small,  approximations  are  obtained  from  reduced  order  models  in 
separate  time  scales.  This  way  the  singular  perturbation  approach  alleviates 
difficulties  due  to  high  dimensionality  and  ill-conditioning  resulting  from 
the  interaction  of  slow  and  fast  dynamic  modes. 

The  chapter  is  organized  as  follows:  In  Section  2,  we  study  the 
fundamental  problem  of  modeling  and  control  of  singularly  perturbed  systems 
driven  by  Wiener  processes  under  various  cases  of  continuous  and  sampled 
observations.  An  extension  of  the  single  parameter  model,  which  realistically 
captures  the  multimodeling  situation,  is  formulated  in  Section  3  using  multi¬ 
parameter  singular  perturbations.  In  Section  4,  we  obtain  multimodel  solutions 
to  Nash  and  team  problems  under  certain  quasi-classical  information  patterns, 
and  establish  their  relationship  with  the  solutions  of  the  full  problem.  We 
summarize  the  results  with  some  concluding  remarks  in  Section  5. 

To  highlight  the  ideas,  we  have  adopted  an  informal  style  for  the 
presentation  and  discussion  of  the  main  results.  More  rigorous  treatment  can 
be  found  in  quoted  references. 


2.  MODELING  AND  CONTROL  OF  STOCHASTIC  SINGULARLY  PERTURBED  SYSTEMS 

2.1.  Well-Posedness  of  Different  Models 

The  optimal  control  of  stochastic  singularly  perturbed  systems  with 
white  noise  inputs  leads  to  difficulties  not  present  in  deterministic  problems. 
This  is  due  to  the  idealized  behavior  of  white  noise  which  "fluctuates"  faster 
than  the  fast  dynamic  variables.  To  illustrate  the  problem  of  optimally 
controlling  a  stochastic  fast  dynamic  system,  consider  the  following  standard 
LQG  formulation 

system  dynamics:  edz  3  Az  dt  +  Bu  dt  +  Gdw  (2.1a) 

measurement  process:  dy  =  Cz  dt  +  dv  (2.1b) 

T 

cost  function:'  J  =  E{zTz  +  /  (z 'Oz  +  u 'u)dt } .  (2.2) 

0 

Here,  e > 0  is  the  small  singular  perturbation  parameter;  w(t)  and  v(t)  are 
standard  Wiener  processes  independent  of  each  other,  and  all  matrices  are 
time- invariant,  with  0,  Q>0.  We  will  further  assume  that  A  is  a  stable 

matrix,  that  is.  Re  A  (A)  <0. 

The  optimal  control  u*  which  minimizes  the  cost  J  is  obtained  in 
the  usual  manner  by  applying  the  separation  principle,  so  that 

u*  =  -B'Kz  (2.3) 

whe.a  K  satisfies  the  Riccati  equation 

eK  =  -A'K-KA-Q  +  KBB’K;  K(T)-^T.  (2.4) 


The  vector  z  (t)  denotes  the  optimal  estimate  of  z(t)  given  the  past  observations 
which  for  any  given  u(t)  is  the  output  of  the  Kalman  filter 
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edz  =  Az  dt  +  Bu  dt  +  PC '  (dy-Cz  dt)  ;  z(0)  =  E[z(0)]  (2.5) 

1 

where  —  P(c)  is  the  error  covariance  of  z(t),  satisfying 

eP  =  AP  +  PA'  +  GO'  -PC'CP;  P(0)  =  eCov(z(0)),  (2.6) 

which  does  not  depend  on  u(t) .  The  resulting  minimum  value  of  the  cost,  J*, 
is  given  by 

T 

J*  =  ez 1  (0)K(0)  z(0)  +  —  tr  [P  (T)  T]  +  -J  tr  [CPKPC  '  +  PQ]dt .  (2.7) 

e  e  0 

Notice  from  (2.6)  and  (2.7)  that  Cov(z-z)  =  0(-^-)  and  J  =0(-^-).  Hence  as  e  +  0, 
both  the  covariance  of  the  estimation  error  and  optimal  cost  diverge,  even 
though  the  feedback  gain  of  the  optimal  control  law  given  by  (2.3)  remains 
finite  (outside  the  end-point  boundary-layer) .  This  is  because,  in  the  limit 
as  e  0 ,  the  fast  variables  z  themselves  tend  to  white  noise  processes,  thus 
losing  their  significance  as  physically  meaningful  dynamic  variables.  Hence 
the  problem  formulation  given  by  (2.1)  and  (2.2)  is  ill-posed.  More  detailed 
analysis  of  this  formulation  in  the  filtering  and  control  context  may  be 
f  ound  in  [5,6]. 

One  way  to  circumvent  the  difficulty  encountered  above  is  to  appro¬ 
priately  "scale"  the  white  noise  terms  in  the  model.  Let  us  now  investigate 
ramifications  of  the  following  more  general  formulation: 

The  state  dynamics  description  is  replaced  by 

Edz  =  Az  dt  +  Bu  dt  +  e^Gdw;  He A (A)  <  0  (2.8a) 

and  the  measurement  process  is 

dy  =  Cz  dt  +  e  dv 


(2.8b) 


where  a, 8  are  some  positive  constants  to  be  chosen.  The  cost  function  J  is 
the  same  as  before. 

Now  the  optimal  control  is  given  by 

u*  =  -B'Kz  (2. 

where  K(t)  satisfies  (2.4). 

The  optimal  estimate  z(t)  is  obtained  from  the  Kalman  filter 

edz  =  Az  dt  +  Bu  dt  +  M(t)  (dy-Cz  dt)  ;  z(0)  =  E[z(0)  ]  (2.1 

where  M(t)  is  the  filter  gain  given  as 

M(t)  =  e1_23PC'  (2.1 

and  P(t)  is  the  error  covariance  of  z(t),  satisfying 

eP  =  AP  +  PA’  +  £2oi-1GG'  -  eU2BPC'CP;  P (0)  =  Cov(z (0) )  .  (2.1 

The  minimum  value  of  the  cost,  J*,  is  given  by 

T  T 

J*  =  ez'(0)K(0)z(0) +tr(P(T)T) +£1-2&J  tr(CPKPC’)dt  +  /  tr(PQ)dt.  (2.1 

0  0 

Let  us  now  examine  the  behavior  of  P(t),  M(t) ,  and  J*  for  various  values  of 
a  and  8,  in  the  limit  as  e-*-0.  The  limiting  behavior  of  P(t)  and  J*  is 
governed  primarily  by  the  parameter  a,  while  the  limiting  behavior  of  M(t) 
is  governed  by  both  parameters  a  and  6.  Notice  that  the  behavior  of  K(t)  is 
unaffected  by  the  scaling. 

A  straightforward  examination  of  (2.12)  reveals  that  for  a<h,  P(t) 
diverges  as  £-*-0,  which  implies  from  (2.13)  that  J*  also  diverges  as  e-^O. 
[Note  that  8 > 0  by  hypothesis.]  When  P(t)  diverges,  the  filter  gain  M(t)  may 
or  may  not  diverge  as  c-*0,  depending  on  the  value  of  6.  If  3  >  h,  however. 
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in  addition  to  0  <  a  <  ^,  M(t)  always  diverges  as  e  +  0.  This  particular  case 
(ct  <h,  8  >  H)  corresponds  to  the  situation  where  the  observations  become  noise- 
free  in  the  limit  as  £->-0,  and  therefore  the  filter  gain  becomes  unbounded. 

When  a  >  h  and  6  is  any  positive  constant,  it  readily  follows  from 
(2.12)  and  (2.13)  that  P(t)  and  J*  go  to  zero  as  £->-0.  If  at  the  same  time 
8  <  h>  then  M(t)  also  goes  to  zero  as  e  +  0.  This  case  (a  >  h,  8  <  h)  corresponds 
to  the  situation  when  the  observations  become  too  noisy  in  the  limit  as 
e->-0,  thus  driving  the  filter  gain  to  zero. 

Hence  the  range  of  scaling  (a,6>0;  a  ^  h,  &^h)  leads  to  ill-posed 
formulations.  This  implies  that  it  is  not  possible  to  give  a  physically 
meaningful  interpretation  to  the  limiting  solution.  [Of  course  for  any  fixed 
e > 0,  the  problem  is  well-defined.]  The  only  meaningful  formulation  is  obtained 
when  a =  8 = h-  In  this  case  P(t),  M(t) ,  and  J*  remain  bounded  and  nonzero  and 
yield  a  well-defined  stochastic  control  problem  in  the  limit  as  e-*0. 

The  above  analysis  has  indicated  that  in  order  to  obtain  a  well- 
defined  stochastic  control  problem,  the  process  and  observation  noise  need  to 
be  scaled  in  an  appropriate  manner.  To  gain  further  insight,  let  us  directly 
examine  the  limiting  behavior  of  the  stochastic  process 

edz  =  Azdt  +  Gdw;  Re  a  (A)  <0,  GG'>0.  (2.14) 

Clearly,  without  the  scaling  term,  z(t)  converges  to  white  noise  in  the  limit 
as  e->-0.  If,  with  the  above  scaling,  z(t)  converges  to  something  which  is 
physically  meaningful,  then  this  would  provide  a  strong  justification  for 
the  model  (2.8),  with  a=h. 

Solving  for  z(t)  from  (2.14)  we  obtain 

2<t>  ,  ;teA(t-t)/eGdw<-) 

,7  o 


(2.15) 


where  we  have  assumed,  without  loss  of  generality,  that  z(0)  -  0.  Now 

Cov(z(t))  =  E{[—  /  eA(t-T?/eGdw(T  )][—  /  eA(t“'9 /eGdw(x9)  ]  ' } 

/e  0  1  /i  0  2 

=I  JteA(t-T)/£GG’eA,(t-T)/edx 
£  0 

-  W£(t)  (2.1 

where  W^(t)  satisfies,  for  each  e  >  0,  the  linear  matrix  differential  equation 

eW  =  AW  +  W  A'  +  GG' . 

e  e  e 

Since  ReA (A)  <0,  we  clearly  have  the  limit  (excluding  boundary  layers) 

lim  Cov(z(t))  =  W  (2.1 

e-*-0 

where  W  is  the  positive  definite  (because  GG'  >0)  solution  of  the  Lyapunov 
equation 

AW  +  WA'  +  GG'  =  0.  (2.1 

This  implies  that  z(t)  converges  in  distribution  to  a  zero  mean  constant 
Gaussian  random  vector  whose  covariance  W  satisfies  (2.18)  [see  also  [7,8]]. 
The  above  convergence  is  indeed  physically  meaningful,  and  therefore  we  are 
justified  in  using  (2.14)  to  model  a  fast  stochastic  dynamic  system. 

Physically,  the  above  analysis  has  indicated  that  in  order  to 
meaningfully  estimate  and  control  a  fast  dynamic  system,  the  influence  of  the 


random  disturbances  has  to  be  "limited"  in  some  sense. 


Singularly  Perturbed  Systems  with  Continuous  Measurements 


Let  us  now  consider  the  full  (with  both  slow  and  fast  variables) 


stochastic  singularly  perturbed  optimal  control  problem 

dx  =  (A^x  +  Aj^z  +  B^u)dt  +  G^dw  (2.19a) 

edz  =  (e^^x  +  A22z  +  B2u)dt  +  ^^dw  (2.19b) 

dy^  =  (C^x  +  C^zHt  +  dv^  (2.20a) 

dy2  =  (eVC21x + C22z)dt  +  eVdv2  (2.20b) 

J  =  Etx'dHjXCO  +2ex’(T)ri2z(T)  +  ez '  (T) T2z(T) 

T 

+  /  (x,L[L1x  +  2e<Sx,L[L2z  +  e2<Sz'L^L2z  +  u,u)dt}.  (2.21) 


The  parameters  a,  3,  v,  6  represent  the  relative  size  of  the  small  parameters 
within  the  system,  with  respect  to  the  small  time  constants  of  the  fast 
subsystem.  The  inclusion  of  a  separate  observation  channel  y ^  for  the  fast 
subsystem  is  essential,  since  otherwise  for  a  >  0  the  fast  variables  cannot  be  esti¬ 
mated  meaningfully  from  the  slow  observation  channel  (signal-to-noise  ratio  tends 
to  zero).  The  stochastic  processes  w(t),  v^(t)  and  V2(t)  are  standard 
Wiener  processes  independent  of  each  other  and  the  Gaussian  random  vector 
[x(0),z(0)].  We  also  assume  that  ReX(A22)  <0.  The  optimal  solution  to  the 
problem  posed  by  (2. 19)— (2.21)  can  be  obtained  by  invoking  the  separation 
principle: 


u*  = 

-((B[K1  +  B^2)x+  (B^K2  +  eBjK12)z] 

(2.22) 

dx  = 

(Aj^x  +  A^2z  +  B^u*)dt  +  [P^C^  +  e01  Vp12C2^da 

(2.23a) 

edz  = 

(eSA-.x  +  A-„z  +  B  u*)dt  +  ea[eP.' 9C  '  +  ea  VP  C’]da 

(2.23b) 

where  the  innovations  process  o(t)  is  defined  by 


A  ^  i  '^11 
do  (t)  «  1  -•  1  1 


e‘Vdy2  [C21 


c,  ,  |  i  * 


A  ""I 

3 

-V  . 

e  dy. 


-  [CL  e“VC2]  _  dt. 


CJ4) 


The  control  gain  matrices  satisfy 


-*!  ■  KlAll  +  e8K12A21  +AilKl  +  ‘**21*12  +  SL1  -  (K12B2  +  K1B1>  (BiKI  +  BiKl2>  ; 


K^T)  -  Tl 


(2.25a) 


-eKi2  -  K1A12  +  Ki2A22  +  eAilK12  +  e8A21K2  +  e  L1L2  "  (K12B2  +  KlV  (B2K2  +  eBlK12)  5 

K12(T)  “  ri2  (2.25b) 

-eK2  =  K2A22  +  A22K2  +  eK12Al2  +  eA12Kl2  +  £:  L2L2  "  (K2B2  +  eKl2Bl^  <B2K2  +  eBlK12^  ; 

K2(T)  =  r2.  (2.25c) 


The  filter  covariances  satisfy 


“  All^+PlAil+£aA12Pl2+eapi2A;2+€lGr(PlCi+£°t'Vpi2C2) 


PL(0)  =  Cov(x(0) ) 


(2.26a) 


eAllPl2+£  Al2P2+P12A2+e  aplA21+ClG2'('PlCi+e  P12C2^  ^eClP12 

+£a"Vc2P2);  Pl2(0)  = eaCov(x(0),z(0))  (2.26b) 

A22P2+P2A22+£l~a+6A21P12+el-a+8pi2A21+G2G2~^ePl2Ci+e:a_Vp2C2^  ^eGlP12 

+ea_VC2P2);  P2(0)=e1-2a 


Cov(z(0)).  (2.26c) 


Ln  order  Co  have  a  finite  cost.  r urtheraore ,  a  well-defined  t orraulat ion  also 

requires  chat 

0  <  a  •  v  i  i  ,  lj.  (2.29) 


The  restriction  a  »  v  is  crucial,  otherwise,  either  the  fast  variables  are  not 
observed  due  to  very  noisy  observations  (a  >  v)  ,  or  they  are  observed  noiselessly 
(a  <  v)  in  the  limit  as  e -*■().  If  a  >  %,  the  problem  becomes  deterministic  as 
e-*0,  and  if  B  >  *5,  the  coupling  between  x  and  z  becomes  negligible.  The 
constraint  6  >  a  insures  that  the  state  z  is  predominantly  fast,  and  relaxing  it 
causes  no  conceptual  difficulties. 

Note  that  when  a=B  =  v  =  0,  it  is  required  that  6  =  \  to  yield  a 
finite  cost.  In  this  case  the  fast  variables  are  of  no  interest  as  far  as 
the  control  is  concerned,  and  serve  only  as  a  model  for  a  wide-band  disturbance 
to  the  slow  variables.  The  important  case  is  when  a  *  v  = \  and  6=B=0,  since 
this  results  in  a  full  weighting  of  the  fast  variable.  For  this  problem,  it 
can  be  shown  that  [9], 


-K  =  K  A  -B  R  N  L  )  +  (A  -B  R  N  L  ) 'K  +  L'(I-N  R  N')L 
o  oooooo  ooooo  o  o  ooo  o 


-K  B  R_1B'K  ; 

O  O  O  0  0 


K  (T)  =  r 
o 


(2.33) 


P  =  A  P  +  P  A'-(P  C'+G  D')V-1(C  P  +D  G')+G  G ' ;  P  (0)  =  Cov(x(0))  (2.34) 

o  oo  oo  ooo  o  o  oooo  oo  o 


A  =  A. . ,  =  B.-A.9A“Jb,,  N  a  -l.a"Jb0,  L  =  L. ,  R  =  N'N 

o  11  o  1  12  22  2  o  2  22  2  o  1  o  o  o 

C  -  C.-C^Ja.,  ,  D  A  -C0A"Jg,,  G  ^  G.  ,  V  =  I  +  D  D ' 
o  1  2  22  21  o  2  22  2  o  1  o  o  o 


(2.35) 


Uf  =  ~B2K2Sf 


(2.36) 


ed£f  =  (A22zf+B2uf)dt  +  P2C'2{dy2-C22£fdt-/F[C21-C22A22A21]isdt 


+  C22A22B2Usdt} 


(2.37) 


K2A22+A22K2  +  L2L2-K2B2B2^2  =  ° 


(2.38) 


A22B2  +  B2A22  +  G2G2-P2C2C2f2  “  °' 


(2.39) 


Notice  that  ug  and  are  obtained  on  solving  a  reduced-order  slow  control 
problem  and  an  infinite-time  fast  control  problem,  respectively.  These  problems 
can  be  solved  independently  of  each  other.  It  is  interesting  to  note  that  the 
fast  filter  is  driven  by  the  slow  variables  as  well.  Hence  the  implementation 
of  the  filters  is  not  independent,  but  sequential  in  nature.  The  near¬ 
optimality  result  (2.30)  is  valid  only  for  t€  (0,T) ,  because  the  boundary- 
layer  terms  have  been  neglected. 


2.3.  Singularly  Perturbed  Systems  with  Sampled  Measurements 


So  far  we  have  examined  the  modeling  and  control  aspects  of 
stochastic  singularly  perturbed  systems  when  the  measurement  process  is 


\ 

■  -4 
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continuous  in  time.  We  shall  now  examine  the  same  aspects  when  the  measure¬ 
ment  process  consists  of  discrete  samples.  Two  types  of  sampled  observations 
will  be  considered.  In  the  first  case,  sampled  values  of  the  state  in  addi¬ 
tive  noise  are  observed,  and  in  the  second  case  sampled  values  of  a  continuous¬ 
time  measurement  process  are  observed.  These  types  of  observations  play  an 

important  role  in  multi-agent  decision  problems  as  we  shall  see  later. 

It  is  a  well-known  fact  that  the  open-loop  dynamics  of  any  system  of 
the  form  (2.19)  with  a=}5,  6=0,  can  be  transformed  into  a  block-diagonal  form 
where  the  pure  slow  and  fast  variables  are  explicitly  displayed  [10].  Hence, 
without  loss  of  generality,  we  shall  assume  that  the  system  to  be  controlled  is 
given  by 

dx  =  (A1x  +  B1u)dt  +  G^dw .  (2.40a) 

edz  =  (A2z  +  B2u)dt  +  /t  G^dw ;  ReX(A2)  <0.  (2.40b) 

The  performance  index  will  be  given  by+ 

T 

J  =  E{x(TK  x(T) +ez’(T)r  z(T)  +  /  (x'Q  x  +  z'Q  z  +  u'u)dt(.  (2.41) 

0 

We  now  consider  two  cases  of  sampled  observations 


2.3.1.  Case  1;  Noisy  measurements  of  sampled  values  of  state 

The  observations  consist  of  sampled  noisy  measurements  of  the  state. 
Specifically,  the  observations 


y(j)  =  C^xUj)  +  C2z(tj)  +  v(j) 


(2.42) 


are  available  at  sampled  time  instant  t.  where  j  =  0, 1 , . . . ,N-1  and 


A  more  general  formulation  would  include  cross  terms  involving  slow 
and  fast  variables.  Here  we  are  avoiding  this  in  order  not  to  obscure  the 
essentials  of  the  following  analysis  by  notational  complexity.  We  should  note, 
though,  that  such  a  restriction  leads  to  no  conceptual  loss  of  generality. 
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^  =  T.  Let  0  =  {0, 1 , . . .  ,N-1 } .  Then  the  random  vectors 
{v(j),je9}  are  assumed  to  have  independent  Gaussian  statistics  with 
v(j)  - N(0,R^) ,  >  0,  j€.9.  Their  statistics  are  also  assumed  to  be  indepen¬ 

dent  of  the  Wiener  process  w(t)  and  the  Gaussian  vector  [x(0),z(0)]. 

A  near-optimal  solution  to  the  problem  defined  by  (2.40)-(2.42) 
can  be  shown  to  be  given  by 


where 

us(t)  =  -B'K14'1(t,t^)xg(t_.)  ;  t£[t^,tj+1),  j£9 

-Kl  »  AJKj  +  KjAj+Qj-^B^JKjJ  K^T)  -  T, 

^(t.t  )  =  (A1-B1B|K1)4'1(t,tj);  ^l.(tj  »tj)  “I 

t€[t  ,t  j),  j£9 

x  (t)  *  A  x  (t)+B  u  (t);  x  (0)=E[x(0)]  \ 

s  is  is  s 

t€  ;  j  =  1,2 . N  \ 

xg(t.)  »  xs(t~)+S1(j)[y(j)-C1xs(t')+C2A’1B2us(t')]  J 

Z  =  A.S  +E  A.’+G.G.';  Z  (0)  *  Cov[x(0)  ]  \ 

s  is  si  ii  s 

t€  t'j-i-tj): 

£s(tj>  •  Js(V-SL(«Cirs(t?  J 


(2.43 


(2.44 

(2. 


(2.46 


(2.47 


(2.48 


(2.49 


A„E,  +  7  A!  +  G»Gl  =  0 


(2.5 


I 


Uf(t>  =  -B^K2f2(t,tj)zf(t,);  t£  A1),  j£e 


J*  j+i' 


a^k2  +  k2a2  +  q2-k2b2b’k2  =  0 


e4'2(t,tj)  =  (A2-B2B'K2)4'2(t,tj);  ^2(tj’tj)  =  1 

te[t,t  ),  j£8 


ezf(t)  =  A,z.(t) +B_u,(t);  z.(0)  =  E[z(0)] 


tG  ttj_i » ;  j“i»2 . n 

Zf  (tj)  =  2f(t-)  +s2(j)[y(j)-C1xs(t-)-C2if(t*)  +C2A21B2us(t^)] 


S2(j)  =  EfC2[ClEs(tj")Ci  +  C2ZfC2+Rj]  * 


It  u*  is  the  optimal  solution  to  the  problem  (2.40)-(2.42) ,  then  it  can  be 


shown  that 


lim  u*  =  un;  0  <  t  <  T 


lim  (J(u*)-J(u  ))  -  0. 
e-0  ° 


2.3.2.  Case  2:  Sampled  values  of  continuous  noisv  measurements 


The  measurement  process  is  a  continuous-time  stochastic  process 


described  by 


y (t)  =  /  [C.x(s)  +C„z(s)]ds  +  q (t)  (2. 

0 

where  q(t)  is  a  standard  Wiener  process  independent  of  w(t)  and  the  Gaussian 
vector  [x(0),z(0)l. 


•l  ■  v *  v  ■  g  p  v  ">  p  >  ».■  ■»"".■ '  'J* 1  'J*  I  ',■''  V? if  1  'ir*  v* 1  *’j  tf 'j  w  v  ■  v  vj  ■  v  ^  r ■  '4  ^ it."  m  u  ^  ■; >■.'».  ■. 1  %.  %  '  1  %  *  T  ’."V  ■  ■v  i 


fi 

I 


:: 


? 


h*  -* 
K*' 


f: 

K 

i 


'•1 
I  i 
\‘ 

»  _ 

*< , 

*  %.' 

v! 

i 
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Let  0  =  t  <  t,  <  ■ • •  <  t„  ,  <  t.  =  T  and  0  =  {0,1,... ,N-1 } .  The  measure- 
o  1  N- 1  N 

ment  process  is  not  observed  on  the  entire  time  interval  [0,T],  but  only  its 
sampled  values  at  time  instants  t^,  ^2****'^  are  0'3Servec^*  Therefore,  the 
only  observation  in  the  subinterval  [t^,t  p  is 


t. 

1 


y(t.)  =  J  [C  x(s)  +  C9z(s)]ds  +  q  ( t . ) 
J  0  1  J 


(2.58) 


which  is  made  at  the  beginning  of  that  subinterval.  In  the  time  interval 
[0,t  ),no  observations  are  made  and  only  the  prior  statistics  of  the  random 
quantities  are  available. 

Let 


y(j)  =  y^)  - 

J 

=  J  [C^Cs) +C2z(s)  ]ds  +  v(j) 

Vl 

where  v(j)  =  q(t.)-q(t_.  is  a  discrete-time  Gaussian  white  noise  process 
with  mean  zero  and  variance  =  (t^-t^  ^)I.  Clearly  the  sigma-algebras 
generated  by  (y(t^) ,i*l,2, . . . , j }  and  (y(i)  ,  i=l,2,...,j}  are  equivalent. 

A  near-optimal  solution  to  the  problem  defined  by  (2.40),  (2.41), 
(2.59)  can  be  obtained  as  follows: 


(2.59) 


U  =  U  +  U- 

o  s  f 


where 


ug(t) 


-BjK1'F1(t.tj)xs(tj); 


te[t.,tj+1),  j£6 


(2.60) 


(2.61) 


& 


r. 


-kj  -  AjK1  +  K1A1+Q1-K1B1B|K1;  Kl(T)  =  Tl 


Mt.tJ  =  (A1-B1B;K1)f.(t,t.);  '^(t4,t4)  =  I 


1  j 


‘1  1117  1 


1  j  J 
t€  1 1  j  .tj+1)  ,  j€0 


(2.62) 


(2.63) 


III 


X  (t) 
s 


A^uJ+B^Ct);  C°)  -  E[x(0)J 


j  =  1»2 . N 


x  (t~) +  S1(j) ty(j)  -  /  [C1xs(r)-C2A~1B2ug(r) ]dr] 

Vi 


E  *  A.  E  +  E  A.'+G.G.' ;  E  (0)  =  Cov[x(0)] 
s  Is  si  11  s 


j  =  1 , 2 , . . .  ,N 


(V  *  es(V-si(3HI  ciVr,tj-i)dr  VVi^'Wi3 

3-1 


t .  t . 

3  J 


+  /  C  /  *  (r,p)G1Gj<>^(t  ,p)dpdr] 
t  r  ^ 

j-i 


t. 

*j  -  JJ  C  <t  (p.t  t)dp  Z  it  )  I2  ^(r.t  )C|dr 

Vl  Cj-1 

t.  t. 

, 3  .P  J 

+  {  Cx  j  <}is(p,r)G1G[  J  <j>gU,r)C|  d2,  dr  dp 

Vi  Vi  r 


,3  rJ 

+  J  C  <f>f(p,t  )dp  Z  J  4^(r,t  )C2dr 

t.  ,  J  t.  .  J 

j-1  1- 1 


t . 

,3  .P  .3  , 

+  J  C2  J  dif(p,r)G2G^  j  $f  U,r)C^d£dr  dp  +  R 

tj-i  Vi  r 


si(3>  -  [<)>s(tj ’cj-,)Es(tj-i)  /  ♦;<r*vi)cidr 

Vi 

t.  t. 

,3  .3  ._i 

+  /  4>  (t  ,p)G1G'  /  $'  (r,p)C|dr  dp]  R 

t,  ,  3  P  J 


4>s(t’ tJ)  =  Al*s(t,tj);  VW  =  1 


te[tj,tj+l)*  j€e 


(2.68) 


e*f(tftj)  =  A2<frf(t,t  ) ;  *f(t  ,£  )  -  I 


t€[tjft  p,  j£9 


Vf  +  EfA2  +  G2G2  =  ° 


uf(t)  =  -B'K  !.(t,t  )z  (t  );  t€[t.,t.x(),  j€  0 


2  2  2'  j  f  j 


j’  j+1' 


(2.69) 

(2.70) 

(2.71) 


a^k2  +  k2a2  +  q2  -  k2b2b2k2  =  0 


(2.72) 


e*2(t,t.)  =  (A,-B,B;K0)V0(tft4);  4'0(t4,ti)  =  I 


2  2  2  2  2  j  ’  2  j  j‘ 


C€[t  ,t  j£9 


(2.73) 


ezf (t) 


A2zf  (t) +B2uf  (t)  ;  zf(0)  =  E[z(0)  ] 


t€[t  ,t);  j  =  1*2, 

J  ^  J 


*N 


(2.74) 


VV 


2f  (t J )  +s2(j)  [y Cj)  -  /  [C1xg(r)+C2zf  (r)-C9A21B2ug(r)  ]dr]  J 

Vl 


S2(j)  =  [<J>f(tj,t^_1)Zf  /  <|>j (r.tj.pc^dr 

"  Cj-1 


t.  t. 

J  r3  .-1 

+  j  <J»f(t  ,p)G2G'  J  <fr'  (r.p)C'dr  dp]  R.  . 

Vi 


(2.75) 


If  u*  is  the  optimal  solution  to  the  problem  defined  by  (2.40),  (2.41),  (2.59), 


then  it  can  be  shown  that 
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lim  u*  *  u  :  0  <  t  <  T 

e+0  o 


lim(J(u*)-J(u  ))  =  0 
e+0  0 


(2.76) 


We  should  point  out  that  the  near-optimality  of  the  composite  control  uq  in 
both  cases  1  and  2  is  valid  only  in  the  open  interval  (0,T)  because  the 
boundary-layer  terms  have  been  neglected. 

An  important  distinction  between  the  above  formulations  involving 
discrete  observations  and  the  earlier  formulation  involving  continuous  obser¬ 
vations  is  that,  in  the  discrete  observations  cases,  there  is  no  need  to 
scale  the  measurement  noise  and  it  is  not  necessary  to  have  a  separate  obser¬ 
vation  channel  for  the  fast  variables.  This  is  because  the  sampling  interval 
is  fixed  and  independent  of  e,  and  hence  there  is  no  interaction  between  the 
dynamics  of  the  observation  process  and  the  input  noise  process. 

Now  that  we  understand  the  subtleties  involved  in  the  modeling  and 
control  of  stochastic  singularly  perturbed  systems  under  various  observation 
patterns,  the  next  step  is  to  study  multi-agent  decision  problems.  But  before 
we  do  this,  we  shall  introduce,  in  the  next  section,  the  important  concept  of 
multimodeling  of  large  scale  systems  within  the  framework  of  time-scales  and 
singular  perturbations.  This  concept  plays  a  crucial  role  in  the  near-optimal 
design  of  multi-agent  decision  policies  for  stochastic  singularly  perturbed 


systems . 
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3.  MULTIMODELING  BY  SINGULAR  PERTURBATIONS 

The  need  for  model  simplification  with  a  reduction  (or  distribution) 
of  computational  effort  is  particularly  acute  for  large  scale  systems  involving 
hundreds  or  thousands  of  state  variables,  often  at  different  geographical 
locations.  Some  form  of  decentralized  modeling  and  control  which  exploits  the 
weak  interactions  between  subsystems  is  then  required.  While  there  are  a 
number  of  approaches  to  the  study  of  large  scale  systems  [1],  the  success  of 
any  proposed  decentralized  scheme  critically  depends  upon  the  choice  of  suo- 
systems  [11]. 

A  fundamental  relationship  between  time-scales  and  weak-coupling 
has  been  developed  for  power  systems,  Markov  chains,  and  other  classes  of 
large  scale  networks  [12-15].  If  the  interactions  of  N-"local"  subsystems 
are  treated  as  0(e),  and  if  each  subsystem  has  an  equilibrium  manifold  (null 
space),  then  the  local  subsystems  are  decoupled  in  the  fast  time  scale.  However, 
they  strongly  interact  in  a  slow  time  scale  and  form  an  aggregate  model  whose 
dimension  is  equal  to  the  number  (N)  of  the  local  subsystems.  The  system 
is  thus  decomposed  into  N+l  subsystems  (N  in  the  fast  and  one  in  the  slow 
time  scale) . 

To  elucidate  this  relationship,  consider  the  following  class  of 
interconnected  subsystems 


dx.  _  N 

=  —  A.,  x.  +  E  A.,  x.  ;  i=  1 , 2 , . . .  ,  N 
dt  e.  li  l  .  ,  it  j  ’  ’  ’  ’ 

i  J  =  1  J 


(3.1) 


where  e^>0  and  A^  is  a  stable  matrix  with  one  zero  eigenvalue.  Assuming  that 

x. (0)  is  not  in  the  null  space  of  A..,  the  first  term  dominates  the  second 
1  n 

term  on  the  right  hand  side  of  (3.1),  and  therefore  the  interconnections  can  be 
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neglected  initially.  As  the  fast  transients  draw  x^(t)  towards  the  equilibrium 
manifold  (the  null  space  of  A^) ,  the  two  terms  on  the  right  hand  side  of  (3.1) 
become  the  same  order  of  magnitude,  and  therefore  from  this  time  onwards  the 
interconnections  can  no  longer  be  neglected.  Hence,  the  dynamic  behavior  of 
(3.1)  can  be  characterized  by  two  separate  motions:  an  initial  fast  transient 
within  each  isolated  subsystem,  followed  by  a  slow  motion  around  the  equilibrium 
manifold  obtained  on  neglecting  the  interconnections.  Therefore,  in  the  short 
term  the  subsystems  can  be  treated  in  isolation,  while  in  the  longer  term  they 
become  strongly-coupled. 


We  now  introduce  a  transformation  to  make  the  slow  and  fast  parts 


of  x^(t)  explicit.  Let 


i  Xl; 


d  x2 


-1 


!Au 


H  V 


dt 


*Ni 
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\tn  M  \ 


0  A12  ...  A1n 

:  A2l  0  •••  A2N 


\l  ^2  •••  ° 


x„ 


*N  i 


(3.2) 


or 


x  =  (SI  LA  +A,)x. 

O  1 


(3.3) 


Define  the  left  and  right  eigenvectors  of  A  for  the  zero  eigenvalue  as 


A  T  =  0,  VA  =0,  VT  =  T 
o  o  N 


T  =  block  diag[t1,t2,.. . ,tN] 


V  =  block  diag[vltv2> . . . ,vN] . 

We  also  define  block-diagonal  matrices  W  and  S  as  follows 
WT  =  0,  VS  =  0,  WS  =  I 

n-N 


Now,  using  the  following  transformation 


x  =  Vx,  x€  R 


N 


z  =  Wx ,  z  £  R 


n-N 


and  its  inverse 

x  =  Tx  +  Sz, 

the  interconnected  system  (3.3)  can  be  transformed  into 

x  =  VAxTx  +  VA^z 

nz  =  fiWA.Tx  +  W(A  +  ftA. ) Sz 
1  o  1 

For  sufficiently  small  e^,  (3.7)  can  be  approximated  by  the  model 
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Notice  that  the  fast  transients  within  the  subsystems  are  decoupled,  and 
they  interact  only  through  the  slow  core.  A  long  term  aggregate  model  is 
obtained  by  letting  and  is  given  by 

x  =  VA.Tx  .  (3.9) 

sis 

The  previous  analysis  has  shown  that  for  a  wide  class  of  large  scale 
systems,  the  notions  of  subsystems,  their  coupling  and  time  scales  are  inter¬ 
related  and  lead  to  a  multiparameter  singularly  perturbed  model  with  a 
strongly-coupled  slow  "core"  representing  the  long  term  system-wide  behavior, 
and  weakly-coupled  fast  subsystems  representing  the  short-term  local  behavior. 

With  the  presence  of  control  and  stochastic  disturbance  inputs,  a 
generalization  of  (3.8)  can  be  obtained  as 

N 

dx  =  A  xdt  +  Z  (A  .z,dt  +  B  .u.dt  +  G  .dw.)  (3.10a) 

oo  j=l  oj  ]  oj  J  cj  j 

N 

e.dz,  =  (A.  X  +  A..Z.+  Z  e . . A . . z .  + B . ,u . ) dt +  /cTg , . dw, 
x  i  io  ii  x  i]  ij  j  li  i  l  ii  l 

i-1,2 . N  (3.10b) 

where  (u^(t);  i=l,2,...,N}  are  the  control  inputs,  and  (w^t);  i=l,2,...,N) 
are  standard  Wiener  processes  independent  of  each  other.  Each  fast  subsystem 
has  its  own  singular  perturbation  parameter  e^,and  is  weakly-coupled  to 
other  fast  subsystems  through  The  fast  subsystem  i  is  affected  by  its 

own  control  input  u^(t)  and  disturbance  input  w^(t).  The  slow  subsystem,  being 
the  common  "core", is  affected,  in  general,  by  all  the  subsystem  controls  and 
disturbances . 

In  a  situation  like  this,  it  is  rational  for  a  subsystem  controller 
to  neglect  all  other  fast  subsystems  and  to  concentrate  on  its  own  subsystem. 
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plus,  of  course,  the  slow  interaction  with  others  through  the  "core."  For 
the  i-th  controller  "to  neglect  all  other  subsystems"  simply  means  to  set 
all  e-parameters  equal  to  zero,  except  for  e^,  which  is  to  be  kept  at  its 
true  value.  The  i-th  controller's  simplified  model  is  then 


e.dz.  =  A.  xidt + A. . z .dt + B . .u .dt +  /eT  G.,dw.  (3.11b) 

1  1  xo  li  l  n  l  i  li  l 

where 

A.  =  A  -  E  A  .aTIa.  ,  B. .  =  B  .  -  A  ,a7*B. . . 
l  oo  oj  22  jo  1J  °J  02  JJ  JJ 

We  denote  x^  with  a  superscript  rather  than  a  subscript  to  stress 
the  fact  that  x1  is  not  a  component  of  x,  but  the  i-th  controller's  view  of 
x.  In  reality,  the  model  (3.11)  is  often  all  that  i-th  controller  knows  about 
the  whole  system.  The  k-th  controller,  on  the  other  hand,  has  a  different 
model  of  the  same  large  scale  system.  This  situation,  called  multimodeling, 
was  first  formulated  and  investigated  in  [2]  in  a  deterministic  setup  (with 
no  disturbance  inputs) . 

In  the  next  section  we  shall  study  the  impact  of  multimodel  assump¬ 
tions  on  the  design  of  multi-agent  control  strategies  in  the  presence  of 
disturbance  inputs  and  noisy  observations. 
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4.  MULTI-AGENT  DECISION  PROBLEMS 


We  shall  restrict  our  discussion  in  this  section  to  the  case  of 
two  decision  makers,  as  this  will  keep  the  notation  simple  and  ease  the 
exposition  of  the  principle  ideas.  All  the  results  that  we  shall  present 
here  extend  to  the  case  of  more  than  two  agents  in  a  fairly  straightforward 
fashion.  Furthermore,  we  shall  present  and  discuss  only  the  main  results;  the 
proofs  of  the  various  propositions  shall  be  omitted,  but  they  can  be  found  in 
the  references  cited. 

It  is  well-known  that  a  system  of  the  form  (3.10)  can  be  transformed 
into  a  system  with  purely  slow  and  fast  variables  [2],  Hence,  without  loss 
of  generality,  we  shall  consider  multi-parameter  singularly  perturbed  systems 
of  the  form 


Eidzi 


A  z  dt  +  l  (B  .u.dt  +  G  .dw  ) 
oo  o  O]  ]  °J  J 


(A. .z . + e . .A. .z. + B. ,u.)dt  +  /eT  G..dw., 
ii  l  ij  lj  j  u  l  l  li  i 


(4.1a) 


i,  j  *1,2;  i^j 


(4.1b) 


with  dimz.=n.,  i  =  0,l,2,and  dimu.=m.,  i=l,2.  The  initial  conditions  are 
li  li 

assumed  to  be  Gaussian  with 

E[ZjL(0)  ]  =  ziQ,  E[z.(0)  zj(0)]  =  Nyj  i,  j  =  0,1,2.  (4.2) 

Furthermore,  we  shall  restrict  ourselves  to  the  case  (Re  A(A^)  <0,  i=l,2}. 

In  a  multimodel  situation,  decision  maker  i  models  only  zq  and  z^, 
but  neglects  z..  Also,  his  observations  are  functions  of  z  and  z.  alone. 
This  situation  with  decentralized  observations  leads  to  problems  involving 
nonclassical  information  patterns,  for  which  no  finite-dimensional  solution 
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exists  in  general.  In  order  to  obtain  finite-dimensional  solutions  which 
can  be  implemented  in  practice,  one  needs  to  modify  the  information  structure. 
In  this  section  we  shall  study  three  problems  with  quasi-classical  information 
patterns.  The  first  problem  is  a  Nash  problem  with  continuous  measurements 
where  the  information  available  to  the  decision  makers  is  restricted  to  the 
state  of  a  finite-dimensional  compensator  of  a  specified  structure.  The  next 
two  problems  are  team  problems  with  sampled  measurements,  where  the  decision 
makers  exchange  information  with  a  delay  of  one  sample  period.  The  two  types 
of  sampled  measurements  are  those  that  we  have  considered  earlier  in  Section  2 


4.1.  Nash  Game  with  Continuous  Measurements 

The  decision  makers  make  decentralized  continuous  measurements  which 
are  given  by 


dy  .  =  C  . z  dt  +  dv  . 
oi  oi  o  oi 


dy .  .  =  C..z.dt  +  /eT  dv..;  i=l,2 

J ii  li  l  i  li 


(4.3) 


where  dim  y  .  =p  .  and  dimy.  =p...  The  processes  v  ,(t)  and  v..(t)  are 
7oi  *01  7ii  *ii  y  ox  xx 


standard  Wiener  processes,  independent  of  each  other  and  of  the  process  noise 
w^t).  Defining  x’  -  [z^  zj  z^l,  y[  =  [y^  ~Z  y^l.  1  »  and 


w'  =  [wj  w^] .  The  system  of  equations  (4.1)-(4.3)  can  be  written  in  a 


composite  form  as 


dx  =  (A(e)x+  Z  B.(e)u.)dt  +  G(e)dw 
i=l  i  1 


(4.4) 


dy^  =  C^(e)xdt + dv^;  i=l,2 


(4.5) 


E[x(0)]  =  x  ,  E[x(0)x'(0)]  =  N 


(4.6) 


where  dim  x  =  n  =  n  -t-n.+n.  and  dim  y  ®  p  *  p  ,+p  .  . .  The  matrices  A(e)  ,  B  .  (e)  , 
o  1  2  i  i  oi  11  l 

G(e),  C^(e),  and  N  are  appropriately  defined. 

The  information  available  to  decision  maker  i  at  time  t  is  given  by 

a^t)  =  {^(t),  XQ,  N}  (4.7) 

where  x^(t)  is  the  state  of  the  n-dimensional  compensator 

dx.  =  (F.x. + H.u.)dt  +  L. [dy .-C.x.dt] .  (4.8) 

i  i  i  l  i  i  i  l 


Let  a^(t)  denote  the  sigma-algebra  generated  by  the  information  set  a^(t). 

Further,  let  denote  the  class  of  second-order  stochastic  processes 

{u.(t),  t>0}  which  are  a .  (t) -measurable.  Then,  a  permissible  strategy  for 

m. 

decision  maker  i  is  a  mapping  :  [0,T]  *!Rn-*-R  x,  such  that  vi(*»  <Oe  HL  . 
Denote  the  class  of  all  such  strategies  for  decision  maker  i  by  r  . 

For  each  r  ;  i*»l,2},  the  cost  functionals  for  the  two  decision 

makers  are  given  by 


wv  ■  EK«>roi‘o<T>+v;<T>ru2i<T> 


T 


i-  1.2 


j-1.2) 

(4.9a) 


or,  equivalently 


J.(V'tVj)  =  E{x' (T)T  .  (e)x(T )+/  (x  'Q  .x  +  u  !u  )dt  |  u.  (t)  ■  v.  (t  ,a.  )  ,  j=l,2} 

1  l  ^  1  Q  111  J  JJ 


i  *  1.2 


(4.9b) 


where  the  expectation  is  taken  over  the  underlying  statistics. 

The  decision  makers  are  required  to  select  the  matrices  F*,  H*,  L*; 


the  initial  conditions  x*(0)  and  strategies  v*[t,x.(t)]  such  that 


Ji(v*,v*)  < 


Vv±e  r.;  i,  j  =  1,2;  i  ^  j 


(A. 10) 


The  pair  of  inequalities  above  defines  the  Nash  equilibrium  point. 

The  optimal  solution  to  the  problem  defined  by  (4 . 1) — (4 . 10)  is 
obtained  by  extending  the  results  of  [16]  to  the  nonzero-sum  case,  and  is 
given  by 


3  3  3 


*  =  -b!k.x. 

i  iii 

;  1=1,2 

(A. 11a) 

[.  -M..)(M 

3  o  Ji  oo 

-Moi)  1] ;  i, j  =  1,2;  i ^  j 

(A. lib) 

*  =  MiiCi; 

i-  1,2 

(A. 11c) 

[*  =  B.; 
i  l 

i  =  1,2 

(A. lid) 

*(0)  =  xq; 

•  i  =  1,2 

(A. lie) 

where  satisfies  the  coupled  set  of  Riccati  equations 

K.  =  -K.A-A'K.  -  Q  +  K. S  K.  +KS.K.  +  K.S.K.;  K.  (T)  -I*, 
i  i  i'i  iii  ijj  JJ1  1  l 

Si  =  BiBi’  =  i  /  j  •  (A. 12) 

M(t)  is  a  symmetric  nonnegative  definite  matrix  satisfying  the  Lyapunov 
equation 

M  =  FM  +  FM'+BB’;  M..(0)=xx'+N,  i  =  j  =  0 

ij  o  o 


=  N 


,  otherwise 


(A. 13a) 
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where 


F  =  A-F*-S„K 


-S2K2 

S1K1 

s2k2 

2K2 

F1"L1C1 

s2k2 

1K1 

S1K1 

F*-L*C 

(4.13b) 


The  compensators  are  unbiased,  in  the  sense  that  for  all  t€  [0,T), 

E{x(t) |x^(t) }  =  xi<t);  i = 1,2. 


Furthermore 


ECtxUJ-x^tJJx^t)}  *  0; 


i  s  1,2. 


(4.14) 


(4.15) 


Thus,  each  component  of  the  error  x(t)-xi(t)  is  orthogonal  to  each  component 
of  x^(t),  and  >L(t)  may  be  regarded  in  some  sense  an  estimate  of  x(t).  Notice 
that  the  solution  exhibits  a  unidirectional  separation  in  estimation  and  control. 
Although  the  control  gains  are  obtained  independently,  the  optimal  filter 
matrices  and  covariance  M(t)  depend  on  the  control  gains,  resulting  in  a 
"dual  effect"  [17], 

The  optimal  costs  are  given  by 


J*  =  x'K.(0)x  +  tr{M.  .(0)K.(0)  +  f  (K.S.K  M,  .  +  K.S.K.M. 
l  ox  o  xi  i  ^viiinijjj( 

+  KjSjKiMoj)dt};  i,  j  =  1,2;  i  *  j  . 


(4.16) 


The  linear  strategy  (4.11a)  is  the  unique  Nash  strategy  for  this  problem. 
Since  the  finite-dimensional  estimators  (4.8)  are  not  Kalman  filters,  it  is 


not  clear,  at  the  outset,  what  their  limiting  structure  (as  the  small  parameters 
go  to  zero)  looks  like.  Does  the  full-order  estimator  decompose  into  a  number  of 
decoupled  low-order  estimators?  Is  it  possible  to  obtain  a  near-equilibrium 
solution  from  low-order  subproblems? 

It  will  be  shown  that,  in  the  limit  as  the  small  parameters  go  to  zero, 
the  full-order  estimator  (4.8)  decomposes  into  an  n^-dimensional  estimator 
in  the  slow  time  scale  which  has  a  similar  structure,  and  two  n^-  and  n£- 
dimensional  Kalman  filters  in  the  fast  time  scale.  Furthermore,  the  near¬ 
equilibrium  solution  is  in  fact  the  multimodel  solution,  i.e.,  the  solution 
obtained  when  decision  maker  i  neglects  ,  and  models  only  zq  and  z^.  The 
multimodel  assumption  leads  to  the  formulation  of  three  low-order  subproblems: 
two  independent  stochastic  control  problems,  one  for  each  decision  maker,  in 
the  fast  time  scale,  and  a  stochastic  Nash  game  in  the  slow  time  scale. 

The  slow  subproblem  is  obtained  by-  neglecting  all  the  small  para¬ 
meters  in  (4.1),  and  is  given  by 

2  2 

dz  =  (A  z  +  E  B  .u.  )dt  +  E  G  dw.  (4.17) 

os  oo  os  oi  is  oi  l 


c  . 

1  0 

oi 

1 

z  dt  +i 

08  i  i  -u 

0 

i  - —  C. .A. .B., 

L  rr  11  11  LiJ 

u.  dt  + 
is 


dv 


oi 


-1. 


dv .  .  -C  .  .  A .  .  G  .  .  dw . 

li  li  li  li  l 


(C .  z  +D.  u,  )dt  +  dv.  ; 
is  os  is  is  is 


i-  1,2 


E[z  (0)]  -  z  ,  E[z  (0)  z '  (0) ]  *  N  . 
os  oo  os  os  oo 


(4.18) 

(4.19) 


Each  decision  maker  is  constrained  to  use  only  an  n^-dimensional  compensator 


of  the  form 


(4.20) 


dz  *  (F.  z.  +  H .  u.  )dt  +  L.  [dy.  -C .  z.  dt-D.  u.  dt] , 
is  is  is  is  is  is  is  is  is  is  is 


i  -  1,2. 


Let 


a  (t)  =  {z  (t),  z  ,  N  } 
is  is  oo  oo 


(4.21) 


and  a.  (t)  denote  the  sigma-algebra  generated  by  the  information  set  a.  (t) . 


Further,  let  H.  denote  the  class  of  second-order  stochastic  processes 


(u.  (t) ,  t>0}  which  are  a.  (t) -measurable .  Define  the  slow  strategy  v.  ,  as 
is  is  is 

n  m. 

the  mapping  \),  :  [0,T]  xR  °-*-R  ,  such  that  v.  (•,<*.  )£  H,  .  Denote  the  class 

is  is  is  is 

of  all  such  slow  strategies  for  decision  maker  i  by  F .  . 


For  each  (v.  €  F.  ;  i=l,2},  the  slow  cost  functionals  for  the 
is  is 

decision  makers  are  given  by 


^is^  Jls’V2s^ 


E{z’  (T)F  .z  (T)+f  (z*  0  .z  +  u !  R .  u .  ) dt 
os  oi  os  Jq  os  oi  os  is  is  is 


|ii.  (t)  «  v  (t,a  )  ,  j-1,2}; 
J b  j  j  s 


i»  1,2 


(4.22) 


where 


R,  =1+  (aT!b.  .)  'Q.  (aTIb.  .)  . 
is  li  li  ^i  ii  ii 


The  decision  makers  are  required  to  select  the  matrices  F*  ,  H*  ,  L* 

n  IS  IS  1! 


the  initial  conditions  z*s(0),  and  strategies  v*g[t,z^s(t) ]  such  that 


J, (v*  ,v*  )  <  J . (v  ,v*  )  Vv.  €  r.  ; 
i  is  js  i  is  js  is  is 


i,j  =  1,2,  i ^  j •  (4.23) 


The  optimal  solution  to  the  slow  subproblem  defined  by  (4. 17)-(4.23)  is 
given  by 

v*  “-rT^'.K.  z.  ;  i  =  1 , 2  (4.24a) 

is  is  oi  is  is 


(4.24b) 


F* 


is 


A  -  B  .R?  B".K.  [I  +  (M  -  M  )(M  -  M  )  ];  i,j=l,2; 

OO  OJ  JS  OJ  JS  jo  J1  OO  Ol 


* 

L. 

is 


[M. .C' .  !  G  . (C . . aT^G.  ) '  (I  +  (C..aTJg  )(C..A.Jg  )"}_1]; 

11  01  [  01  11  li  li  ii  ii  ii  ii  ii  ii 


(4.24c) 


i-1,2 


H.  =  B  .  ;  i=l, 2 

IS  01 


z.  (0)  =  z  ;  i=l , 2 
is  oo 


(4.24d) 
(4 . 24e) 


where  K.  is  the  solution  of  the  coupled  set  of  Riccati  equations 
is 


K  =  -  K  A  -  A'  K  -  Q  .  +  K.  S.  K.  +  K.  S.  K.  +  K.  S .  K .  ; 
is  is  oo  oo  is  voi  is  is  is  is  js  js  ]s  js  is 


(4.25) 


-1. 


Kis(T)  ■  roi  •  Sis  •  BoiRisBoi  !  ‘•J'1-2' 


M(t)  is  a  symmetric  nonnegative  definite  matrix  satisfying  the  Lyapunov 
equation 


M  =  F  M  +  MF"  +  B  B'  ;  M. . (0)=  Iz  z'  +  N  ;  i=j=0 

S  S  S  S  1J  <*  OO  OO  oo 


(4.26a) 


where 


N  ,  otherwise 
*  oo 
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C. .A  1G. .  0 

11  li 


;  i-l,2. 


(4.26b) 


The  optimal  costs  are  given  by 


J.  =  z '  K.  (0)z 
IS  OO  IS  oo 


+  tr{M. . (0)K.  (0)  +  /( K.  S.  K.  M. .  +  K.  S.  K.  M. 

li  is  is  is  is  ii  is  js  js  j< 


+  K.  S.  K.  M  . )dt)  ;  i,j=l,2  ;  i^j 

JS  JS  IS  Oj 


(4.27) 


The  fast  subproblems,  on  the  other  hand,  are  formulated  'locally'  at  the 
subsystem  level.  These  are  stochastic  control  problems  because  the  decision 
makers  do  not  interact  in  the  fast  time  scale: 


e.dz.,  =  (A..z.,  +  B..u.c)dt  +  G. .dw. 
l  lr  n  if  ii  if  l  ii  l 


(4.28) 


dy..c  =  C..z.,dt  +  dv. . 
'nf  ii  if  l  ii 


(4.29) 


E  [  zif  (0)  ]  =  z.Q  ,  E[z.f(0)  z'f(0)]  =  Ni;. 


(4.30) 


J.,  =  E{e.z;,(T)r. .Z. .(T)  +  /( z;.Q..z  +  u;  u  )dt 

if  l  if  ii  if  ifxn  if  it  if 


(4.31) 


Notice  that  this  fast  subproblem  is  exactly  the  one  we  studied  in 
detail  in  Section  2.  Its  solution,  as  cr+Q,  is  given  by 


u  c  =  -  B  T  .K.  ^.z.  . 
if  ii  if  if 


(4.32) 


where  satisfies  the  Riccati  equation 


k.,a. .  +  a:.k.,  +  q..  -  k.cb..b:.k.c  =  o 

if  11  11  if  11  if  11  11  if 


(4.33) 


and  z^  is  the  state  of  the  Kalman  filter 
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z.dz.e  =  (A..Z.,.  +  B..u  )dt  +  P . cC  7  .  [  dy.  _ 
1  if  11  if  li  if'  if  li 1  •'if 


C..z  dt]  ; 

li  if 


Z  .  - (0)  =  2  . 
if  io 


(4.34) 


P^  is  the  error  covariance  of  z  ^  satisfying 


p.,a:.  +  a..p.,  +  g..g:.  -  p._cr.c. .p..  =  o  , 

if  11  11  if  11  11  if  11  11  if 


(4.35) 


and  the  optimal  cost  is  given  by 


J.c  *  T  tr{P.,Q..  +  C^P.-K.-P.-C:.} 

if  if  li  li  if  if  if  li 


(4.36) 


The  following  proposition  establishes  the  connection  between  the  solutions  of 
the  slow  and  fast  subproblems  and  the  full-order  problem.  Its  proof  may  be 
found  in  [ 7  ]  . 

Proposition  4.1: 

i)  v.(t,k.(t))  =  v*s(t,z.s(t))  +  u*f(zif(t))  +  0(||-|)  ;  V  t€(0,T) 

*  *  * 


ii)  =  Jis  +  Jif  +  T  tr{QiiWi>  +  0(| e| )  ;  i-1,2 


where 


e  =  [e1  e2  e±2  ] 

and  is  the  nonnegative  definite  solution  of  the  Lyapunov  equation 


A..W.  +  W.A.\  +  G..GT.  =  0  ;  i=l,2 

li  l  l  li  ii  li 


(4.37) 


Since  the  multimodel  strategies  need  only  decentralized  'state 
estimates,'  each  decision  maker  needs  to  construct  only  two  filters  of 
dimensions  and  n^,  respectively,  instead  of  constructing  one  filter  of 
dimension  nQ  +  +  nn  as  required  by  the  optimal  solution.  This  would 

result  in  lower  implementation  costs. 


35 


4.2.  Team  Problems  with  Sampled  Measurements 

We  shall  now  consider  problems  wherein  the  measurement  processes  of 

the  decision  makers  are  not  continuous  on  the  entire  time  interval  [0,T],  but 

consist  of  sampled  values  observed  at  time  instants  t  ,  t,,...,t  ,  ,  where 

o’  1’  N-l 

0  =  t  <  t,  <...<  t„  ,  <  t,T  =  T.  Let  8  denote  the  index  set  {0,1, . . . ,N-1} , 
o  1  N-l  N 

and  y^(j)  denote  the  p ^-dimensional  observations  made  by  decision  maker  -  i  at 

time  instant  t^ ,  j€0 .  Thus  the  only  measurement  of  decision  maker  -  i  in  the 

subinterval  ft.,t.,,)  is  y.(i). 

j  j+1  7i  J 

The  quasi-classical  information  pattern  that  we  shall  consider  here 

is  the  so-called  "one-step-delay  observation  sharing  pattern,"  wherein  the 

decision  makers  exchange  their  independent  sampled  observations  with  a  delay 

of  one  sampling  interval.  Hence,  the  information  available  to  decision  maker  -  i 

in  the  time  interval  [t.,  t..,)  is 

J  J+1 


'  {yi(J)-  Vi 


(4.38a) 


where,  5.  ,  denotes  the  common  information  available  to  the  decision  makers 
J-l 


in  the  same  interval , % i .e . ; 


5j_1  =  { yx ( j -l) ,  y 2 ( J  — i ) , • • • >y1(°) ,  y2(0)} 


(4.38b) 


Let  denote  the  sigma-algebra  generated  by  the  information  set 

i  n 

x . ,  and  H.  denote  the  class  of  stochastic  processes  (u.(t),  t>0}  whose 


restriction  to  the  interval  [t.,t.,,)  is  a. -measurable  for  all  i€0.  Then 

J  J+1  i  J 


(p.+p7)N 


permissible  strategy  for  decision  maker  -  i  is  a  mapping  \> .  :  [0,T]  x  IR  K1 
mi  N 

■*  IR  ,  such  that  v^(*,a^)GH^.  Denote  the  class  of  all  such  strategies  for 

N  .  N 

decision  maker  -  i  bv  F..  For  each  {v.GF.  ;  i=l,2},  we  define  the  quadratic 

i  li 


strictly  convex  cost  function  as 
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J(v  ,V9)  =  E{z;(T)r  Z  (T)  +  Z  e. z  r (T) r . z . (T) 

12  o  oo  .,11  ll 

1=1 

T  2 

+  /(ZoQ0Zo  +  1  (ziQiZi  +  uiui))dtluj (t)  =  vj (t,aj> ’  J=1»2} 


(4.39a)  - 


where  {r\>0,Q^>0,  i=0,l,2}  and  the  expectation  operation  is  taken  over  the 
underlying  statistics. 

Equivalently,  in  terms  of  the  composite  state  vector  x(t)  of  (4.4), 
the  cost  function  can  be  written  as 

T 

J(v1,v2)  =  E(x'(T)r(e)x(T)  +  /(x'Qx  +  u£u  +  u'u  )dt|u  (t)  =  v.(t,a.),  j  =1 , 2 } 

0  J  J  J 

(4.39b) 

where  F(e)  and  Q  are  appropriately  defined  in  terms  of  the  matrices  appearing 


in  (4 . 39a)  . 


A  team  optimal  solution  is  a  pair  (v^  ,  i=l,2}  which  satisfies 


&  * 

J(v  ,v?)  =  inf  inf  J(v  ,v  ) 
rN  N 
1  2 


(4.40) 


Here  optimal  and  near-optimal  strategies  .will  be  obtained  for  two  cases  of 
sampled  observations,  as  delineated  below. 


4.2.1.  Case  1:  Noisy  measurements  of  sampled  values  of  state 


At  sampled  time  instant  t^.j^Q,  the  decision  makers  observe 


y.(j)  =  C.  z  (t.)  +  C..z.(t.)  +  v.(j) 
l  J  10  o  J  11  1  j  1 


=  Cix(tj)  +  vj,(j)  ;  i=l,2 


(4.41) 


The  random  vectors  {v^ ( j ) ; j€0 , i=l , 2 }  are  assumed  to  have  independent  Gaussian 
statistics  (v^(j)  ~  ^(0,  V „  )  ,  >0,  j£9  , i=l, 2 } .  Their  statistics  are  also 


V  /  V 

•  •  -  «  «  •  »  *. 


assumed  to  be  independent  of  the  Wiener  processes  {v^Ct)  ;  i=l,2}  and  the 


initial  state  vector  x(0). 

The  optimal  team  solution  to  the  problem  defined  by  (4.4),  (4.6), 
(4 . 38) -(4 . 41)  has  been  derived  in  [18],  and  is  given  by 


vi(t,ai)  =  Pi(t)[yi(j)  -  C.l(j)]  -  B'S(t)i^(t,tj)5(j)  ;  i=l,2 


te[t.,t.+i)  ;  jee 


(4.42a) 


where  P^(t),  P£(t)  are  piecewise  continuous  functions  on  [0,T]  and  satisfy 


the  coupled  set  of  linear  integral  equations 

t 

pi(t)  =  B'S.(t)  /  ^.,(t,t)B.B'L  (t)di  -  B.L  (t)  ;  i-1,2 


t . 
3 


,tj+i)  ;  jee 


(4.42b) 


where 


i- 

Lij  (t)  =  Si(t)  [  ( t ,  tj )  +  /  o(t,T)BkPk(T)dTCk]Zi(j)  +  K„(t); 


t . 
J 


i ,k=l , 2  ;  i/k  ;  t€[t^,t^+1)  ;  j£0 


(4.42c) 


K^t)  =  -  (A  -  B.B'S.(t))'Kij(t)  -  Si(t)Bkpk(t)CkI.j  ;  i,k«l,2,  i#k 


Kij(tj+1}  =  °  *  t€[tj  ’  tj+l]  ’  jG0 


(4 . 42d) 


S(t)  and  S^(t)  satisfy  the  Riccati  equations 


S  =  -  A'S  -  SA  -  Q  +  S[B1B1'  +  B  B']S  ;  S(T)  =  F 


(4.42e) 


S.  =  -  A'S.  -  S.A  -  Q  +  S.B.B.'S.  ;  S.(t.)  =  S(t.) 
l  i  i  i  i  l  i  i  j  j 


tS(t,  ,,tj  ,  i=l ,  2  ,  i=N . 1 


(4 . 42f ) 


Tvv 


!  V  Pf  I  "■  J ' 


-  ••  -. '- '  -*-  v^r^v' "-  ■’v  vmr  ■-'"  -- '  "-^  v  wr 


¥■ 
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» 


^(t,x)  is  the  state  transition  matrix  satisfying 


l-  ■  -  • 


i^(t,t)  =  (A  -  B1B"S  -  B^S^U.x)  ;  ^(t,t)  =  I 
^ij(t»T)  is  the  state  transition  matrix  satisfying 

i.jCt.r)  =  (A  -  BiB^Si)4-.j(t,t)  ;  *...(x,x)  =  I  ;  teU..,^) 


(4.43a) 


m 


kr-'x' 


i-1,2  ,  jee 


+>(t,x)  is  the  state  transition  matrix  satisfying 
Kt,x)  =  A4>  ( t ,  x  )  ;  <J>  (x  ,  x)  =  I 
C(j)  *  n(t~)  =  E [ x( t  ) | 6 . __ ]  and  n(t)  satisfies 

■J  J  J  "*■ 

•  ^  * 

n  *  An  +  E  B1vi(t,a^)  ;  n(0)  =  x_ 


i=l 


t€[t  ^.t  )  ,  j-1, . . .  ,N 

n(t.)  =  n(t  )  +  M(j)[y(j)  -  Cn(t7)] 

J  J  J 

Vj)  =  ^(t^c'tc.ECt.)^  +  V..]"1  ;  i-1,2  ,  jee 
where  E(t7)  =  E[(x(t^)  -  n(tj))(x(tj)  -  n(t"))'  ]  and  E(t)  satisfies 


E  =  AE  +  EA'  +  GG'  ;  E (0)  =  N  ) 
t€[tj-X’tj)  5  3=1’--'>N 


E(t.)  =  E(tj)  -  M(j)CE(t~) 


(4.43b) 


(4.43c) 


(4.44) 


(4.45) 


(4.46) 


and 


K-. 


M(j)  =  E ( t ^ ) C ' [CE ( 1 7) C  '  +  V  ] 


-1 


(4.47a) 
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VJ  ■  dii>s(W 

(4.47b) 

y(j)  =  fy{(j) 

(4.47c) 

C  =  [C£  c']' 

(4.47d) 

Due  to  the  presence  of  widely  separated  eigenvalues,  the  integro-diff erential 
equations  (4.42)-(4.47)  involved  for  computing  the  optimal  solutions  are 
numerically  stiff.  This  renders  the  optimal  solution  computationally 
infeasible,  specially  when  the  order  of  the  system  is  very  large.  Futhermore, 
when  the  small  perturbation  parameters  are  unknown,  or  when  one  decision  maker 
does  not  have  a  knowledge  of  the  fast  dynamics  of  the  other  decision  maker, 
it  is  not  even  possible  to  compute  the  optimal  solution.  Hence,  there  is  a 
need  to  look  for  suboptimal  solutions.  The  multimodel  solution  proposed 
below  exploits  the  special  structure  of  the  system  to  yield  a  solution  which 
does  not  require  a  knowledge  of  the  small  parameters,  and  allows  the  decision 
makers  to  model  only  their  own  fast  dynamics.  More  importantly,  as  in  the 
problem  with  continuous  measurements,  the  multimodel  solution  is  well-posed 
in  the  sense  that  it  is  the  limit  of  the  optimal  solution  as  the  small 
parameters  go  to  zero. 

The  multimodel  solution  is  obtained  on  solving  three  low-order 
problems:  a  slow  team  problem  under  the  one-step-delay  observation  sharing 
pattern,  and  two  fast  stochastic  control  problems,  one  for  each  decision 
maker . 

The  system  model  for  the  slow  subproblem  is  given  by  (4.17),  (4.19) 


and  the  observations  by 
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7i.U>  ■  Cio*o.(V  +  Vi<j) 


=  y^j)  -  Ci£Zis (t j )  ;  j£0  ,  i-1.2  . 


(4.48) 


The  cost  function  is  given  by 


T  2 

J  (v,  ,v0  )  =  E{z"  (T)F  z  (T)  +  j(z'  Q  z  +  Z  ul  R.  u.  )dt|u.  (t) 
s  Is  2s  os  o  os  i  os^o  os  is  is  xs  1  ]S 

0  i=l  J 


=  v .  (t,a  )  ,  j=l,2> 
J  53  J 


(4.49) 


where 


R.  =  I  +  (AT^B.  .  )  'Q.  (aT'I'B.  . ) 
is  v  li  ii  xi  ii  ii 


The  optimal  solution  to  the  slow  team  problem  defined  by  (4.17),  (4.19), 
(4.48)  and  (4.49)  is  given  by 


v*(t,a.)  «  P,.  (t)[y,  (j)  -  C.Jjj)]  -  R7V.S (j)  ;  i-1,2 


is  i  is  w  is 


is  oi  s  s  j  s 


tSlti,Vi)  ;  jS9 


(4.50a) 


where  pls(t),  p2s(t)  satisfy  the  coupled  set  of  linear  integral  equations 


P13(C>  ■  Rli8oiSis(t>  /  *iJs<t'T)BoiRUBoiLijs<T)dT  -  RIsBoiliJS(t)  •  ‘-1'2 

J 


tettrtj+i>  :  jse 


(4.50b) 


where 


LiJS(t)  ■  +  /  ‘s(t’t),.kEi;ipkiwd,cko|iis® +  Kijs' 


i,k=l,2  ;  i^k  ;  tG[t^.,t^+1)  ;  j€0 


(4.50c) 
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K,  .  (t)  =  -  (A  -  B  ,rT1B'.S.  (.t)rK,  .  (t)  -  S.  (t)B  .rT1?,  (t  )C,  Z .  (j)  ; 
ijs  oo  oi  is  01  is  ijs  is  ok  ks  ks  ko  is  J 

K, .  (t  )  =  0  ;  i,k=l,2  ;  i^k  ;  t€  (t  ,t  j  ;  j^e  (4.50d) 
ijs  j+l  J  j+lJ 


S  (t)  and  S  (t)  satisfy  the  Riccati  equations 
s  is 

S=-A'S-SA  -  Q  +  S  [B  1r71B%  +  B  _r71B'0]S  ;  S  (T)  =  T 
s  oo  s  s  oo  o  s  ol  Is  ol  o2  2s  02  s’  s  c 


(4.50e) 


S.  =  -  A"  S .  -  S.  A  ,  -  Q  +  S.  B  .  R.  B  ' .  S  .  ;  S.  (t.)  =  S  (t.)  , 

is  oo  is  is  ol  o  is  oi  is  oi  is  is  j  s  j 


t^(t^_^,t^]  !  i  1 , 2  ;  j  N, . . . ,1 


(4.50f ) 


( t , t )  is  the  state  transition  matrix  satisfying 


■Jj  (t,t)  =  (A  -  B  .R^B'  S  -  B  0R„^B  '~S  )ip  (t,t)  ;  ^  (x,t) 
s  oo  ol  Is  ol  s  o2  2s  o2  s  s  s 


(4.51a) 


^ijs(t’T)  is  the  state  transition  matrix  satisfying 


V..  (t,i)  =  (A  -  B  .R.  B'S,  )ip..  (t,t)  ;  ip .  .  (t,t)  =  I 
ijs  oo  oi  is  oi  is  ijs  ijs 


t£[tj ,t^+1)  ;  i=l,2  ;  jee 


(4.51b) 


4>  (t,t)  is  the  state  transition  matrix  satisfying 


4>g(t ,t)  =  Aoo<6s(t,T)  ;  4>g(x,t)  =  I 


(4.51c) 


£  (j)  =  n(t  )  =  E [z  (t . ) 1 6 .  J  and  n  (t)  satisfies 
s  s  J  OS  J  1  J-l  s 


n  =  A  n  +  Z  B  .v.  (t,a.)  ;  n  (0)  =  z 
s  oo  s  oi  is  i  s  oo 


t€  [t^._^,tj)  !  j  1 ,  -  .  .  ,N 


(A. 52) 


ns(V  =  ns(tj)  +  Ms(j)[ys(j)  '  Cons(tj)3 


E.  (j)  =  E  (t.)cr  [C.  E  (t.)C'  +  C, .W.cr.  +  V.  .]  ;  i-1,2  ;  jee  (4.53) 

is  J  s  j  10  10  s  j  10  ii  1  ii  13  ’  J 

where  satisfies  (4.37). 

E  (t.)  =  E[(z  (t  )  -  n  (t.))(z  (t.)  -  n  (t.))']  and  E  (t)  satisfies 

S  J  OS  J  S  J  OS  J  S  J  b 


E  =  A  E  +  E  A'  +  E  G  .G' .  ;  E  (0)  =  N 

S  OO  S  S  00  .  ,  Ol  Ol  s  oo 

1=1 

tS[t^_^,tj)  j  j— 1,...,N 


(4.54) 


E  (t.)  =  E  (t.)  -  M  (j)C  E  (t.) 
s  j  s  j  s  J  o  s  2 


m  (j)  =  e  (t.)c'tc  e  (tT)c'  +  e  c..w.c:.  +  V.]' 

s  J  s  i  ol  o  s  1  o  .  ,  11  1  11  j J 

J  J  i=l  J 


ys(j)  =  Cyls(j)  y2s(j)] 


c  =  [c:  c:  ] 

o  lo  2o 


Cn  =  [cn  0] 


C22  *  [0  C225 


(4.55a) 


(4.55b) 


(4.55c) 


(4 . 55d) 


(4.55e) 


The  fast  subproblem  for  decision  maker  -  i  is  defined  by  the  system  equations 
(4.28),  (4.30),  the  observations 


'  cjzif(tj>  +  vi<3) 


5  y.(j)  -  C  z  (t.)  -  C..z.  (t.)  ;  j€0 
'  l  J  io  os  j  ii  is  j 


(4.56) 


and  the  cost  function 


j  -  E(£i.;,«)vlf(i)  +  /<zuVi£  +  • 


(4.57) 
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Notice  that  we  have  studied  this  stochastic  control  problem  earlier  in  Section  2. 
Its  solution,  as  e^-+0,  is  given  by 


u,f  =  -  B;.K.f*.,(t,t.)z..(t.)  ;  tG[t . , t . , , )  ;  j€0 


11  if ylf  J  1J  J 


y  i+r 


(4.58) 


where  satisfies  the  Riccati  equation 


A . K .  ,  +  K.  -A.  .  +  Q.  -  K.  ,B  .  .BT. K.  ,  =  0 
li  if  if  li  xi  if  li  ii  if 


(4.59) 


^^(t.t^)  is  the  state  transition  matrix  satisfying 


e.^.r(t,t.)  =  (A..  -  B.  .Bj.K.,)iJj.«(t,t.)  ;  ^.,(t.,t.)  =  I 
i  if  j  v  ii  ii  ii  if  if  j  ’  rif  j  ’  j 


te[t.,tj+i)  ;  jee 


(4.60) 


z^  is  the  output  of  the  filter 


Gizif  “  Aiizif  +  Biiuif  ’  te^ fcj -1 ’ C j ^  ’  j-1’2’’ 


zif(0)  =  zio 


(4.61) 


;if(t.)  =  zif (t~)  +  M.f(j)[yif(j)  -  C..zif(t')] 


M.-(j)  =  w.cr.tc.  i  (t.)cr  +  c..w.cr.  +  v..]' 

if  J  i  ii 1  10  s  j  10  ii  i  ii  ij 


(4.62) 


The  following  proposition  establishes  the  near-optimality  of  the  multimodel 
solution.  Its  proof  may  be  found  in  [19]. 

Proposition  4.2: 

i)  vi(t,ai)  =  vig(c.ai)  +  uif(t)  +  °(|e|D  !  VtG(O.T)  ;  i=l,2 


ii)  J(vrv2)  =  Js(v-lg,v2g)  +  Z  [Jif(u.f)  +  T  tr(QiW.)]  +  0(|efi) 


Mote  that  in  the  time  interval  ft  ,t,)  no  observations  are  made  and  the 

o  1 

decision  makers  have  access  only  to  the  prior  statistics  of  the  random 
quanities  involved.  Here,  {q^(t)  ;  i=l,2}  are  standard  Wiener  processes 
independent  of  each  other.  Furthermore,  their  statistics  are  also  assumed 
to  be  independent  of  the  Wiener  processes  fv^(t)  ;  i=l,2}  and  the  initial 
state  vector  x(o) . 

Let 

y^j)  =  )  -  yj/j-i) 

t . 

=  /  Cix(t )dx  +  vi(j)  ;  i=l , 2  (i 

Ci-1 

where  v^(j)  =  q^Ct^)  -  q C t ^ is  a  discrete-time  Gaussian  white  noise 

process  with  zero  mean  and  variance  V.  .  =  (t.  -  t  ,  )I. 

ij  J  J-l 

Let  be  given  by  (4.38)  with  y^(j)  replaced  by  y^(j),  and  let 
denote  the  sigma-algebra  generated  by  al .  Then  clearly,  o^  and  are 
equivalent . 

The  optimal  team  solution  to  the  problem  defined  by  (4.4),  (4.6), 


(4 . 38) -(4 . 40)  and  (4.64)  can  be  obtained  in  a  manner  analogous  to  Case  1, 
and  is  given  by  [20] 
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=  P  (t)[y  (j)  -  /  C  n(T)dt]  -  B'S(t)iKt,t  )C(j)  ;  i=l,: 
C3-l 


;  J€e 


(4.65a) 


where  P^(t),  P„,(t)  satisfy  the  coupled  set  of  linear  integral  equations 

t 

P.(t)  =  B'Si(t)  /  *  (t,i)BiB'L  (T)dx  -  B'L  (t)  ;  i-1,2 


;  jee 


(4.65b) 


where 


Li:j(t)  =  Si(t)>(t,tj)Z.(j)  +  Si(t)  f  ♦(t,T)BkPk(x)dT  Ai(j)  +  ( t)  ; 


i,k=l,2  ;  i#k  ;  t€[tj,t^+1>  ;  jSQ 


(4.65c) 


K..(t)  -  -  (A  -  BiB'Si(t))'Klj(t)  -  S.(t)BkPk(t)A.(j)  ;  Ki:j(tj+1)  =  0 

i»k*l,2  ;  i#k  ;  t€(tj,tj+1]  ;  jG@  . 


(4.65d) 


S(t)  and  S^(t)  satisfy  the  Riccati  equations  (4.42e)  and  (4.42f),  respectively. 
The  state  transition  matrices  ij)(t,x),  if/„(t,t)  and  <J)(t,x)  satisfy  the 
equations  (4 . 43) . 


5(j)  =  n(tj)  =  E[x(tj) | ^j_^]  and  n(t)  satisfies 


n  =  An  +  Z  Bivi(t,ai)  ;  n(0)  = 
i=l 


t£[t.  ^,t.)  j  j=l) • • • »N 
J  Cj 

n(t  )  =  n(t  )  +M(j)[y(j)  -  /  Cn(x)dx] 


(4.66) 


Zi.(J)  and  AiO)  are  appropriate  dimensional  matrices  defined 


by 


c  j  c  j  t . 

Zi(j)  -  /  <t>  '(t,t  )Cfdt  +  /  <p(t  ,r)GG'  f  <p'(T 

cj-i  b-i 


;-i 


•  C^didrJV  ;  i-1,2  ;  jee 


j  t- 

Vj)  =  [/  C^^(t,t  )dtZ(t  )  fJ  r(t,t.  .)crdt 

t.  .  J  L 

J-l  Cj-1 


t . 

.J 


t . 
J 


+  /  C.  ,(t  ,r)GG'  /  d'(t,r)crdTdr]v7l  : 

b-1  ' 


i,k=l,2  ;  i^k  ;  j€=6 


/here 


rj  Cj 

'lj  *  I  ( t ,  c  )  dcZ  ( t .  .  )  /  ,)crdt  + 

h-i  cj-i  : 


V.  . 
1-3 


t . 

3  r 


J  i  j 

f  /  ci  /  4»(r,r)GG'  /  <j> '  (£  ,t  )C  (dididr  ;  i-1,2  ;  j€0 

t~  *-  1 


C3-l  'j-l 


(tj)  E[(x(t^)  -  n(tj  ))  (:<(tj  )  -  n  ( t  j  )  )  '  ]  and  E(t)  satisfies 


‘V 


l  =  AI  +  ZA"  +  GG "  ;  1(0)  =  N 

t€[tj_lft  )  ;  j=l, ... ,N 

t.. 

2(t  )  -  M(  j )  [  /  Cl(r,t  )dr£(t  )<T(t.,t 

'  i-  3  3  1-33  ~  1 

3-1 


t .  t . 

J  J 

+  /  /  Cf  (r ,  i  )GG  " -p  '( t .  , t ) dxdr  ] 

^-1  r 


*r) 

(4.67a) 

(4.67b) 

(4.67c) 


(4.68) 


*VA3>V> 


M(j)  is  given  by 


As  in  Case  1,  the  optimal  team  strategies  are  unique  and  linear  in 
the  information  available  to  the  decision  makers,  but  the  expressions  involved 
are  more  complicated.  Hence,  the  computational  problem  worsens,  making  the 
need  for  suboptimal  solutions  more  acute.  Again  the  appealing  structure  of 
the  multimodel  solution  makes  it  an  attractive  alternative. 

As  in  earlier  problems,  the  multimodel  solution  is  obtained  on 
solving  a  lower  order  team  problem  in  the  slow  time  scale  and  two  low  order 
decentralized  control  problems  in  the  fast  time  scale.  The  system  model  for 
the  slow  subproblem  is  given  by  (4.17),  (4.19),  the  cost  function  by  (4.49), 


and  the  observations  by 
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^is(j)  =  /  Cio2os(T)dT  +  Vi(j) 

Cj-1 


=  y. (j)  -  /  C..z.  (x)dx  ;  i=l,2  ;  jGs-{o} 

i.  11  X3 

Vi 


(4.71) 


The  optimal  team  solution  to  this  slow  subproblem  is  given  by 

t . 

v.  (t,a.)  =  P.  (t)[y.  (j)  -  /  C.  n  (i)di]  -  R.  B'.S  p  (t,t.)£  (j)  ; 
is'  1  is  iyis  J  J  10  s  IS  Ol  S  S  J  s  J 


i.1,2  ;  te[t  ,tJ+1)  ;  JS6 


(4.72a) 


where  P^g(t)  ,  P^^t)  satisfy  the  coupled  set  of  linear  integral  equations 


P.  (t)  =  R? B'.S.  (t )/  p,  .  ( t , i ) B  .R7  B'.L..  (x)dt  -  R.  B'.L..  (t)  ; 
is  is  oi  is  J  ijs  oi  is  oi  ijs  is  01  ijs 

J 

i*i.2  ;  te[t^,t^+1)  ;  jee 


(4.72b) 


where 


Lijs(t)  =  Sis(t)$s(t’Cj)Eis(j)  +  Sis(t)^  ,Js(t>T)BokRkXs(T)dTiis(j)  +  Kijs(t) 


i»k=l,2  ;  i^k  ;  t€[t^.,t^+1)  ;  jS0 


(4.72c) 


K.  .  (t)  -  -(A  -  B  .rTJb'.S.  (t) )'  K .  .  (t)-S.  (t)B  ,  R^P,  (t)A.  (j)  ; 

ijs  oo  oi  is  oi  is  ijs  is  ok  ks  ks  is  J 

Kijs(tj+1)  =  0  ’  i’k=1’2  ;  i?*k  ;  t€(tj  ,Cj+l^  ’  (4 . 72d) 


Ss(t)  and  S^g(t)  satisfy  the  Riccati  equations  (4.50e)  and  (4.50f),  respectively. 

The  state  transition  matrices  P  (t,x),  p..  (t,x)  and  p  (t,x)  satisfy  the 

S  1J  s  s 

equations  (4.51).  Furthermore, 


^„(j)  =  n  (t  )  =  E[z  (t  )|<5  ,]  and  n  (t)  satisfies 

°  a  J  os  j  J  “A  s 


n  = 


A  n  +  -  B  .V.  (t,a.)  ;  n  (0)  =  z 

oo  s  ,  ,  01  is  ’  l  ’  s  oo 

i=l 


\(tj) 


t . 

3 


n.(t  }  +Ms(J)[VJ)  -/  Cons(i)dx] 


Cj-1 


(j)  and  A  (j)  are  appropriate  dimensional  matrices  defined  by 


(j)  =  [<J>g(t 


t . 
.1 


t . 
J 


j’tj-i)£s<tj-i)^  ♦s<t-tj-i)ci°dt  +  /  Wr)  £  Goi 
j-1  Cj-1  1_1 


G' 
oi  oi 


t 

,J  —1 

J  ^s(T,r)CiodTdr]Vij  i=1’2  ;  je0 


t . 
J 


t . 
J 


(j)  =  [/  C  *  (t,t  )dtZ  (t  )/  «r(t,t 

ei-i  0  J  *j-i 


i>cLdt 


+  /J  Ck»s(t  r)  j  c  G'  /}'(,, rjc^dxdr]  v'1  ; 
tj_^  o  1=1  r  J 


i,k=l,2  ;  i^k  ;  j€9 


where 


t . 
3 


*  Vlj  +  l  )/ 

j-1  b-1 

!■)  j 

+  l  cil*U<c-c}-l  >dtWif  *i£(t,ti-l>Ciidt 

C  .  ,  t  J 


,)c:  dt 

1  io 


(4.73) 


(4.74a) 


(4.74b) 


+  j  J  C.  4>  (t  ,r)  E  G.  Gr  U'(H,t)c:  d£dxdr 
J  1  10  s  .  ,  10  io;  s  10 

t  .  ,  t  .  ,  1  =  1  T 

J-l  J-l 


t.  t. 

J  f  J 

+  /  /  Cii<p±f(T  ,r)G_.i.G'if<p'f(Z,T)C'idZdTdr  ;  i=l,2  ;  j€d 


t.  ,t.  , 

3-1  J-l 


'ii  iiJYif' 

i 


(4.74c) 


is  the  state  transition  matrix  satisfying 


e.$.c(t,t.)  =  A.  .  <j> .  r  ( t ,  t . )  ;  $.,(t.,t.)  =  I 
i  if  ’  j  ii  if  ’  j  if  j  j 


t€[t^.,t^+1)  ;  i=l,2  ;  jSe 


(4.74d) 


and  satisfies  (4.37).  Now, 


E^zos<tj>  -  ns<tJ»<*os(tj>  -  ns(tj})']  =  W 


where  E  (t)  satisfies 
s 


E  =  A  E  +  E  A'  +  EG  .G".  ;  E  (0)  =  N 
s  oo  s  s  OO  .  ,  01  Ol  s  oo 

1=1 


t^ [t^^^,tj)  ,  j  1,...,N 


(4.75) 


t.  t. 

.3  r3  2 

+  J  J  C  4>  (r,x)  E  G  .G ' .  $ ' (t .  ,x)dxdr  ] 

t.  .  r  i=l  J 
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Mg(j)  is  given  by 


M  (j)  =  1 4>  ,)f  (t.  ,)[  <p'(r,t.  ,)C'dr 

s  J  Lys'  j*  i~l'  s'  j-l  J  s  j-1  o 

J-l 

t.  »  t. 

.J  2  J  _-l 

+/  £  G0iGol/»s(r'T)CodrdT|Vj  : 

*1-1  1-1 

j€0 


where 


c . 
J 


t . 
.J 


V.  V.  +  /  C  <j>  (t,t.  .  )dtl  (t.  .)/  <T(t,t.  ,  )C'dt 

1=1  i  oys'  j-l'  s'  i-l/Jk  s'  j-l'  o 


Ci-1 


*J-1 


tj  r  2  Cj 

+  /  /  C  <0  (t  ,r)  I  G.  G r  /V(*.,T)C'd£dTdr 

J  j  OS  ’  .  ,  10  ioJ  so 

t.  .t.  -  1=1  T 

1-1  1-1 


2  'j  - 


+i=lft  ^ii^if (t,tj-l)dtWi^  *if (t,tj-l)Cii 
Cj-1  tj-l 


dt 


t.  t. 

1  r  ,1 

+  /  /  ci;L<i>if  (x  ,r)GiiG'i/(!)'f  («,,T)C;Tid«,dTdr]  ;  j€e 

t .  .  t .  .  T 

1-1  1-1 

V.  is  defined  by  (4.70a);  Cq,  C^,  ^22  are  defined  by  (4.55c-e)  and 


ysQ>  =  ty{s(i)  y£s(J)  1 


(4.76a) 


(4.76b) 


(4.77) 


The  fast  subproblem  for  decision  maker  -  i  is  defined  by  the  system  equations 
(4.28),  (4.30),  the  cost  function  (4.57)  and  the  observations 


yif(j)  =  /  ciizif(T)dT  +  vi(j> 


=  y.  (j)  -  /  [C.  z  (t)  +  C .  .  z  .  (t)]c1t  ;  j* 

J X  J  10  os  IX  IS 


This  control  problem  has  been  studied  earlier  in  Section  2.  Its  solution. 


as  e^-K),  is  given  by 


u.r  =  -  B  T  .K.  ,4; .  c  (t ,  t . )  z  .  ,(t . )  ;  tS[t.,t.,-)  ;  jS9 
if  11  ifrif  ’  2  if  J  J  J+l 


where  satisfies  the  Riccati  equation  (4.59)  and  ^j.f(t,tj)  satisfies 


(4.60) . 


zif  is  the  output  of  the  filter 


e.z  =  A..z  +  B..u  ;  t€[t.  ,,t.)  ;  i=l,...,N 

i  if  li  if  li  if  ’  1  j-1’  j  J 


z .-(0)  =  z. 
if  10 


zif(t  )  -  z  (tT)  +  M  f(j)[y  f(j)  -  /  C1±z  f(T)dr] 

h-i 


tlf(j>  -  *rt(r,Cj.1)C-1dr 


+  /  *if(tj-T)<!iiCiV  +if(t-T)C£ldrdTl''iJ 


A  near-optimality  result,  analogous  to  Proposition  4.2,  can  be  established  in 


this  case  also  by  following  the  same  lines: 
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Vis(t,Cti)  +u*f<t)  +  0(1  eD  ?  Vt€(0,T);  i-1,2 
2 

Js(vls’v2s)  +  iE1[Jif(uif)  +  T  tr(0i«Wi)l  +  °(1£1> 


5 .  CONCLUSIONS 

Through  an  informal  discussion  of  a  series  of  problems, we  have  attempted 
to  analyze  the  interaction  between  model  simplification  and  strategy  design  in 
a  multimodel  context.  The  objective  was  to  achieve  a  clear  understanding  of  the 
interrelationships  between  the  structural  features  of  large  scale  systems,  like 
time-scales  and  weak-coupling,  and  strategy  design  under  certain  quasi-classical 
information  patterns. 

Weakly-connected  subsystems  with  continuous  equilibria  exhibit  a 
two-time-scale  behavior.  The  slow  system-wide  behavior  is  caused  by  the 
interconnections  and  is  described  by  an  aggregate  'core'  which  appears  as  a 
slow  subsystem  in  the  singular  perturbation  form  of  the  model.  The  fast 
phenomena  which  consist  of  'local'  transients  within  each  subsystem  are  weakly- 
coupled.  The  control  design  problem  for  such  systems  can  be  approached  via 
the  multimodeling  concept.  Each  decision  maker's  control  can  be  divided  into 
a  slow  part,  which  contributes  to  the  control  of  the  core,  and  a  fast  part 
controlling  his  own  fast  subsystem.  Hence,  the  slow  subproblem  is  a  multiple 
decision  maker  problem  under  the  same  solution  concept  (Nash,  team,  etc.)  as 
the  full  problem,  while  the  fast  subproblems  are  decoupled  stochastic  control 
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problems.  Since  each  decision  maker  need  not  know  the  parameters  associated 
with  the  fast  subproblem  of  other  decision  makers,  the  multimodel  solution 
is  robust  with  respect  to  modeling  errors;  a  very  desirable  feature  in  large 
scale  system  design. 

Our  results  serve  to  demonstrate  the  richness  in  the  modeling 
structure  with  multiparameter  singular  perturbations  in  the  context  of 
multimodeling  problems.  In  each  case,  the  limit  of  seemingly  complex 
integro-dif ferential  equations  associated  with  the  optimal  solution  has  a 
nice  appealing  structure  when  interpreted  as  a  multimodel  solution.  Thus 
the  multimodeling  approach  using  singular  perturbations  is  in  some  sense 
'robust'  with  respect  to  a  class  of  solution  concepts  and  information  patterns. 
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